Papers with retrieval speed
Compressing Sentence Representation for Semantic Retrieval via Homomorphic Projective Distillation (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing pre-trained language models produce large sentence embeddings, resulting in performance gap between large and small models. |
| Approach: | They propose a method that augments a small Transformer encoder model with learnable projection layers to produce compact sentences while mimicking a large pre-trained language model to retain the sentence representation quality. |
| Outcome: | The proposed method achieves 2.7-4.5 points performance gain on STS and SR tasks while maintaining the quality of the pre-trained language models. |
Generative Semantic Hashing Enhanced via Boltzmann Machines (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for generative semantic hashing assume a factorized posterior distribution, enforcing independence among the bits of hash codes. |
| Approach: | They propose to use a Boltzmann machine distribution as the variational posterior to introduce correlations among the bits of hash codes. |
| Outcome: | The proposed method can achieve significant performance gains by combining two hash codes. |
Integrating Semantics and Neighborhood Information with Graph-Driven Generative Models for Document Retrieval (2021.acl-long)
Copied to clipboard
Zijing Ou, Qinliang Su, Jianxing Yu, Bang Liu, Jingwen Wang, Ruihui Zhao, Changyou Chen, Yefeng Zheng
| Challenge: | Existing methods for document hashing combine only one of semantics and neighborhood information, lacking a theoretical principle to guide the integration process. |
| Approach: | They propose to encode neighborhood information with a graph-induced Gaussian distribution and integrate it with generative models. |
| Outcome: | The proposed model can be trained as efficiently as state-of-the-art methods on benchmark datasets. |
Efficient Nearest Neighbor Emotion Classification with BERT-whitening (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for retrieval-based methods use pre-trained models for similarity search, but they perform poorly in retrieving semantically similar sentences. |
| Approach: | They propose a non-parametric emotion classification method using nearest neighbor retrieval that can be added to any pre-trained model without further training. |
| Outcome: | The proposed method improves the pre-trained model by 1.17 F1-macro on two emotion classification datasets. |
TRIAL: Token Relations and Importance Aware Late-interaction for Accurate Text Retrieval (2025.emnlp-main)
Copied to clipboard
| Challenge: | Late-interaction based multi-vector retrieval systems rely on a naive summation of token-level similarity scores . this leads to inaccurate relevance estimation due to tokenization of semantic units and the influence of low-content words. |
| Approach: | They propose a late-interaction-based multi-vector retrieval system that uses token relations and token importance in relevance scoring. |
| Outcome: | Extensive tests show that TRIAL achieves state-of-the-art accuracy compared to existing methods. |